An accurate estimate of camera orientation with respect to a specific reference, such as the vehicle supporting the camera, facilitates many image processing and computer vision functionalities. Such functionalities include, but are not limited to, view synthesis (e.g., generating images from different viewing perspectives using a single backup camera, stitching images from surround-view cameras to form a panoramic image, etc.), object detection (e.g., detecting lanes, detecting obstacles on the road, etc.), and three-dimensional (3D) reconstruction (e.g., recovering the 3D structure of the scene using stereo matching from a pair of cameras or using structure from motion from a single camera), which are common components in advanced driver assistance systems (ADAS). A camera's orientation may be represented using several interchangeable representations, which include but are not limited to Euler angle representation (i.e., pitch, yaw, and roll angles following a predefined rotation order), rotation matrix representation, quaternion representation, and axis-angle representation.
Conventional camera orientation estimation uses specially-designed patterns, such as grid patterns, and time-consuming and error-prone techniques. Additionally, if the camera position changes, such as when the vehicle supporting the camera gets into an accident, the camera must be re-calibrated with special equipment and procedures to re-estimate the camera orientation.
Installing inertial measurement units (IMUs) on a camera has limited applicability due to the diminished precision and durability of consumer-grade IMUs and the high cost of higher-grade IMUs.
Other solutions for automatic camera orientation estimation without specially-designed patterns generally employ conventional feature detectors and feature matchers that either directly use local image patches or rely on high-dimensional feature vectors extracted from those patches to describe and match features. These feature detectors and feature matchers generally require the use of a frame buffer to hold an image, which consumes a great deal of power and adds a substantial cost to design. Techniques employing model fitting methods that heavily rely on linear system solvers also require more storage and power, which lead to a higher cost calibration system. Techniques estimating only one or two rotation angles generally assume that the rest of the rotation angles are known and remain constant. This assumption limits the applicability of such techniques.
Given the limitations of existing calibration methods and the limitations of IMUs, a low cost, low-power consumption calibration method with the capability of automatic camera orientation estimation without specially-designed patterns is desirable. Such capability is herein referred to as self-calibration.